Image dehazing and restoration

ABSTRACT

Methods for dehazing a digital image and for restoring an underwater digital image. The methods include the following steps: First, clustering pixels of a digital image into haze-lines, wherein each of the haze-lines is comprised of a sub-group of the pixels that are scattered non-locally over the digital image. Second, estimating, based on the haze-lines, a transmission map of the digital image, wherein the transmission map encodes scene depth information for each pixel of the digital image. Then, for a hazy image, calculating a dehazed digital image based on the transmission map.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 16/092,0553 filed onOct. 8, 2018, which is a National Phase of PCT/IL2017/050426 filed Apr.6, 2017, which claims priority to U.S. Provisional Patent ApplicationNo. 61/319,338, filed Apr. 7, 2016, entitled “Image Dehazing andRestoration”. The contents of the above applications, are incorporatedherein by reference in their entirety.

BACKGROUND

The invention relates to the field of digital image processing.

Outdoor digital images often suffer from low contrast and limitedvisibility due to haze, fog, or other atmospheric phenomena. As usedherein, the term image means a digital image as collected by a digitalcamera sensor from photons emitted by a physical scene, and stored on anon-transitory computer-readable storage medium. The digital imagecomprises pixel, where each pixel has three or more color channelvalues. Haze, for example, results from small particles in the air thatscatter the light in the atmosphere. Fog results from tiny waterdroplets suspended in the air near the earth's surface. Haze and fog areindependent of scene radiance and have two effects on the acquiredimage: they attenuate the signal of the viewed scene, and introduce anadditive component to the image, termed the ambient light, or airlight(the color of a scene point at infinity). As used herein the termairlight means a set of values representing color, such as the red,green, and blue color channel values, that represent the color of thehaze in the images where no objects are visualized. The imagedegradation caused by haze or fog increases with the distance from thecamera, since the scene radiance decreases and the airlight magnitudeincreases. Thus, hazy or foggy images may be modeled as a per-pixelconvex combination of a haze/fog-free image and the global airlight.

Images taken in media other than air may suffer from similardegradation. In addition, some media, such as water, is characterized bywavelength-dependent transmission, distorting the colors in theresulting image. Compensating for this wavelength-dependent transmissionmay require applying different attenuation coefficients for differentcolor channels in the image. This is sometimes done with underwaterimagery.

The foregoing examples of the related art and limitations relatedtherewith are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the figures.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems, tools and methods which aremeant to be exemplary and illustrative, not limiting in scope.

One embodiment provides a method for dehazing a digital image,comprising: operating at least one hardware processor to: cluster pixelsof a digital image into haze-lines, wherein each of the haze-lines iscomprised of a sub-group of the pixels that are scattered non-locallyover the digital image; estimate, based on the haze-lines, atransmission map of the digital image, wherein the transmission mapencodes scene depth information for each pixel of the digital image; andcalculate a dehazed digital image based on the transmission map.

Another embodiment provides a method for restoring a digital image,comprising: operating at least one hardware processor to: convert adigital image of an underwater scene to a plurality ofmedium-compensated images that are each based on attenuation coefficientratios of a different water type; and for each of the plurality ofmedium-compensated images: (a) cluster pixels of the medium-compensatedimage into haze-lines, wherein each of the haze-lines is comprised of asub-group of the pixels that are scattered non-locally over themedium-compensated image, (b) estimate, based on the haze-lines, atransmission map of the medium-compensated image, wherein thetransmission map encodes scene depth information for each pixel of themedium-compensated image, and (c) calculate, based on the transmissionmap and the attenuation coefficient ratios, a restored digital image ofthe underwater scene.

In some embodiments, the clustering of pixels comprises: representingcolors of the pixels of the digital image in a spherical coordinatesystem whose origin is an estimated global airlight value: uniformlysampling the unit sphere of the representation, to output a plurality ofcolor samples each associated with one of the pixels of the digitalimage; grouping the color samples based on their θ (Theta) and φ (Phi)angles in the spherical coordinate system, according to a mutual closestpoint on the unit sphere, thereby producing multiple groups each beingone of the haze-lines.

In some embodiments, the clustering of pixels comprises representing thecolor differences between the pixels of the digital image and theairlight value on a pre-computed tessellation of the unit sphere, wherethe pre-computed tessellation is uniformly sampled and stored inCartesian coordinates in a KD-tree. The clustering of pixels comprisessearching for nearest neighbors on the KD-tree using Euclidean distancecoordinates. The clustering of pixels comprises grouping the colorsamples based on the nearest neighbors, thereby producing multiplegroups each being one of the haze-lines.

In some embodiments, the estimating of the transmission map comprises:estimating an initial transmission map as the quotient, for eachindividual pixel of the pixels of the digital image, of: (a) a distanceof the individual pixel from an airlight value, and (b) a distance froma pixel which is farthest away from the airlight value and belong to thesame haze-line as the individual pixel; regularizing the initialtransmission map by enforcing a smoothness of the digital image on theinitial transmission.

In some embodiments, the estimating of the transmission map comprises:estimating an initial transmission map as the quotient, for eachindividual pixel of the pixels of the digital image, of: (a) a distanceof the individual pixel from an veiling-light value, and (b) a distancefrom a pixel which is farthest away from the veiling-light value andbelong to the same haze-line as the individual pixel; regularizing theinitial transmission map by enforcing a smoothness of the digital imageon the initial transmission.

In some embodiments, the method further comprises operating said atleast one hardware processor to: for each of the restored digitalimages: (a) perform global white balancing of the restored digitalimage, to output a white-balanced image, (b) calculate a standarddeviation of a red channel of the white-balanced image and of a greenchannel of the white-balanced image; and output the white-balanced imagehaving the lowest standard deviation.

In some embodiments, the method further comprises computing theestimated global veiling-light value by: generating an edge map of thedigital image; thresholding the edge map, to produce multiple pixelblobs; and determining that a color or an average color of pixels makingup a largest one of the multiple pixel blobs, is the globalveiling-light value.

Another embodiment provides a system that comprises: an image sensorconfigured to acquire the digital image of any one of the embodimentslisted above; a non-transitory computer-readable storage medium havingstored thereon program instructions to perform the steps of any one ofthe embodiments listed above; and at least one hardware processorconfigured to execute the program instructions.

A further embodiment provides a computer program product comprising anon-transitory computer-readable storage medium having program codeembodied therewith, the program code executable by at least one hardwareprocessor to perform the steps of any one of the embodiments listedabove.

Another embodiment provides a method for estimating a set of airlightcolor channel values for a digital image. The method comprisingoperating at least one hardware processor to automatically perform themethod actions. The method comprising an action of receiving a digitalimage comprising a plurality of pixels, each pixel comprising at leastthree color channel values. The method comprising for each of theplurality of pixels, an action of assigning, based on the color channelvalues, a Hough transform vote for each of the plurality of pixels to atleast one of a plurality of candidate airlight color channel value sets,each of the sets comprising at least three airlight color channelvalues. The method comprising, based on the assigned votes, an action ofselecting one of the sets as the airlight color channel value set of thedigital image.

In some embodiments, each pixel color channel value and each airlightcolor channel value is one of a red channel value, a green channelvalue, and a blue channel value.

In some embodiments, the assigning comprises computing for each pixel aplurality of distances, in a color channel value space, between eachpixel and a plurality of candidate haze-lines, wherein each of theplurality of candidate haze-lines is defined by (a) one of the pluralityof candidate airlight color channel value sets and (b) one of aplurality of solid angles. The assigning comprises comparing theplurality of distances with an adaptive threshold, wherein the adaptivethreshold is based on the distance from each pixel to the respective oneof the plurality of candidate airlight color channel value sets. Theassigning comprises, for each pixel, assigning at least one vote to someof the plurality of candidate airlight color channel value sets based onthe comparison.

In some embodiments, for each Hough transform vote, the at least one ofthe plurality of candidate airlight color channel value sets that isvoted for, is brighter than the voting pixel.

In some embodiments, the method further comprises selecting, for eachpixel, a plurality of subsets, each subset a unique combination of atleast two color channel values, thereby producing at least three limitedcolor channel datasets. The method further comprises performing thesteps of assigning and selecting for each of the at least three limitedcolor channel datasets, producing at least three selected airlight colorchannel value sets. The method further comprises combining the at leastthree selected airlight color channel values to produce a singleairlight color channel value set.

In some embodiments, the method further comprises grouping the pluralityof pixel color values into a plurality of clusters, wherein the vote isassigned for each of the plurality of clusters.

In some embodiments, the plurality of clusters are grouped by at leastone of a k-means algorithm and a Minimum Variance Quantizationalgorithm.

In some embodiments, the assigned vote for each of the plurality ofclusters is weighted by a statistical parameter of each respectivecluster.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thefigures and by study of the following detailed description.

BRIEF DESCRIPTION OF THE FIGURES

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application with color drawing(s)will be provided by the U.S. Patent and Trademark Office upon requestand payment of the necessary fee.

Exemplary embodiments are illustrated in referenced figures. Dimensionsof components and features shown in the figures are generally chosen forconvenience and clarity of presentation and are not necessarily shown toscale. The figures are listed below.

FIG. 1 illustrates the haze-lines prior;

FIGS. 2 a-2 c show a comparison between haze-lines and color lines;

FIGS. 3 a-3 d illustrate a validation of the haze-lines prior;

FIG. 4 shows an airlight-centered spherical representation;

FIGS. 5 a-5 b show distance distribution per haze-line;

FIGS. 6 a-6 h show intermediate and final results of the presentdehazing technique;

FIGS. 7 a, 7 b, and 7 c (i)-7 c(ii) show a comparison of the presentdehazing technique and previous methods, on natural images;

FIGS. 8 a-8 d show a comparison of transmission maps and dehazed imagesof the present dehazing technique versus previous methods;

FIG. 9 illustrates the advantages of the present, global, approach overa patch-based approach;

FIG. 10 illustrates color clustering;

FIG. 11 illustrates attenuation coefficients of Jerlow water types;

FIGS. 12 a-12 f demonstrate the effect of scattering medium on observedscene colors;

FIGS. 13 a-13 c demonstrate the importance of water type matching;

FIG. 14 shows a comparison of the present restoration technique andprevious methods, on natural images;

FIG. 15 . shows a comparison of the transmission maps of the presentrestoration technique and previous methods;

FIGS. 16 a-16 d show a comparison of the present restoration techniqueand previous methods, on a JPEG-compressed input image;

FIG. 17 shows an exemplary underwater image and a veiling light pixelblob computed for that image;

FIGS. 18 a-18 f show an exemplary selection of airlight values fromHough transform;

FIGS. 19 a-19 d show images used for comparison of selection of airlightvalues computed with Hough transform and with alternative techniques;

FIGS. 20 a-20 h show a comparison of selections of airlight values fromHough transform for images with and without visible sky; and

FIGS. 21 a-21 d show a comparison of dehazing between Hough transformand an alternative technique.

DETAILED DESCRIPTION

Disclosed herein is single-image dehazing technique that operatesglobally on a hazy image without having to divide the image intopatches. The technique relies on the assumption that colors of ahaze-free image are well approximated by a few hundred distinct colors,that form tight clusters in red-green-blue (RGB) space, such as a threevalues where each represents the intensity of that color channel. A keyobservation of the present application is that pixels in a given clusterare often non-local, i.e., they are spread over the entire image planeand are located at different distances from the camera. In the presenceof haze, these varying distances translate to different transmissioncoefficients. Therefore, each color cluster in the hazy image becomes ashape (such as a line, arc, curve, and/or the like ie combination) inRGB space, that is termed here a “haze-line”. For example, the RGBvalues of the cluster pixels are substantially along a line (such assubstantially colinear) extending through the airlight (orveiling-light) RGB value. Optionally, the correlation coefficientbetween a model shape and the pixel color values may be used todetermine the haze-line. Optionally, the haze line model may be loosely(i.e. elastically) associated with the airlight point, rigidlyassociated with the airlight point (i.e. constrainted fitting), and/orthe like. Using these haze-lines, the present technique recovers boththe distance map and the haze-free image. The technique is linear in thesize of the image, deterministic, and requires no training. It performswell on a wide variety of images and is competitive with otherstate-of-the-art methods.

Also disclosed is an adaptation of the single-image dehazing techniquewhich makes it suitable for scenes characterized by wavelength-dependenttransmission, such as under water. The adapted technique takes intoaccount the different attenuation coefficient for the different colorchannels, affected by the medium in which the imaging takes place.

Further disclosed are techniques for airlight RGB value determinationfrom single images.

Image Dehazing

The disclosed technique aims to recover, out of a hazy image, the RGBvalues of a haze-free image. Another, optional, aim is to recover thetransmission (the coefficient of the convex combination) for each pixel,which provides a precursor to the scene depth. These are ill-posedproblems that have an under-determined system of three equations and atleast four unknowns per pixel, with inherent ambiguity between haze andobject radiance.

For simplicity of discussion, the present technique is referred to as“dehazing”, and such terminology is used throughout the specification.However, the technique may also apply to foggy images, underwaterimages, and/or the like. For example, the atmospheric phenomena of hazeand fog are similar in how they affect photographs. Accordingly, it ishereby intended that the term “haze”, and any grammatical inflectionsthereof, is interpreted as relating to haze, fog, or any like imagedegredation due to light's reflection, refraction, scattering,absorption, dispersion, and/or the like.

The technique uses the observation that colors of a haze-free image maybe well approximated by a few hundred distinct colors, as presented, forexample, by M. T. Orchard and C. A. Bouman. Color quantization ofimages. Signal Processing, IEEE Transactions on, 39(12):2677-2690, 1991.This implies that pixels in a hazy image may be modeled by lines in RGBspace that pass through the airlight coordinate. These lines are termedhere haze-lines, to stress this characteristic. Pixels along a haze-linecome from objects that have similar radiance colors, located over theentire image plane. These objects may be located at different distancesfrom the camera. Since their acquired color may be modeled by a convexcombination of the radiance color and the airlight color, such objectsmay span a line in RGB space. We use these lines to estimate theper-pixel transmission based on the pixel's position along the line itbelongs to.

As opposed to recent state-of-the-art methods, the present technique isglobal and does not divide the image to patches. Patch-based methodstake great care to avoid artifacts by either using multiple patch sizesor taking into consideration patch overlap and regularization usingconnections between distant pixels. In the present application, thepixels that form the haze-lines are spread across the entire image andtherefore capture a global phenomena that is not limited to small imagepatches. Thus, our prior is more robust and significantly more efficientin run-time.

The present technique is an efficient algorithm that is linear in thesize of the image. We automatically detect haze-lines and use them todehaze the image. Also presented here are the results of extensiveexperiments conducted by the inventors to validate the technique andreport quantitative and qualitative results on many outdoor images.

We first present the haze model and then describe how we use non-localhaze-lines for image dehazing.

The common hazy image formation model, as discussed in W. E. K.Middleton. Vision through the atmosphere. Toronto: University of TorontoPress, 1952, is:I(x)=t(x)·J(x)+[1−t(x)]·A,  Eq. (1)where x denotes the image coordinate, I denotes the observed hazy imageRGB values at x, t(x) denotes the transmission at x, and J denotes thetrue RGB radiance of the scene point imaged at x. The airlight A denotesa single color (i.e. RGB values) representing the airlight in imageareas where t=0. It should be emphasized that, although the term“airlight” implies that open air photography is involved, its concept isnonetheless applicable to underwater photography, where it may be termed“veiling light”. In this disclosure, these terms may be usedinterchangably.

To estimate the veiling light, we assume an area without objects isvisible in the image, in which the color of the pixels is determined bythe veiling light alone. This is a reasonable assumption when the lineof sight is horizontal. It does not hold when photographing a reef wallup close, or when the camera is pointed downwards. However, in thesecases, the distance of objects from the camera usually varies less thenin horizontal scenes, and a simple contrast stretch is likely to besufficient.

Optionally, in order to detect the pixels that belong to the veilinglight, we generate an edge map of the image using an edge detectiontool, such as, for example, the Structured Edge Detection Toolbox (P.Dollar and C. L. Zitnick. Structured forests for fast edge detection. InProc. IEEE ICCV, 2013; available online at:https://github.com/pdollar/edges, last viewed Mar. 27, 2017) andthreshold the edge map, to produce multiple connected components (i.e.,multiple pixel blobs). We then look for and determine the largestconnected component. The pixels belonging to the largest connectedcomponent are classified as veiling-light pixels (x∈VL). An example maybe seen in FIG. 17 , where the bottom frame shows only the veiling lightpixels of the top image. The veiling-light A is the color of thesepixels, or, when the thresholding yielded multiple colors, the averagecolor of these pixels.

The scene transmission t(x) is distance-dependent:t(x)=e ^(−βd(x))  Eq. (2)where β denotes the attenuation coefficient of the atmosphere and d(x)denotes the distance of the scene at pixel x. Generally, β is wavelengthdependent and therefore t is different per color channel, as discussedin S. G. Narasimhan and S. K. Nayar. Chromatic framework for vision inbad weather. In Proc. IEEE CVPR, 2000, and in Y. Y. Schechner, S. G.Narasimhan, and S. K. Nayar. Instant dehazing of images usingpolarization. In Proc. IEEE CVPR, 2001. This dependency has been assumednegligible in many previous single image dehazing methods, to reduce thenumber of unknowns. We follow this assumption. The transmission t(x)acts as the matting coefficient between the scene J and the airlight A.Thus, per-pixel x, Eq. (1) has three measurements I(x) and fourunknowns: J(x) and t(x), resulting in an under-determined estimationproblem.

The present technique, as briefly discussed above, is based on theobservation that the number of distinct colors in an image is orders ofmagnitude smaller than the number of pixels, as presented, for example,by Orchard et al. (1999), Id. This assumption has been used extensivelyin the past and is used for saving color images using indexed colormaps.The present inventors have validated and quantifies it on the BerkeleySegmentation Dataset (BSDS300), available online athttp://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/,last viewed Apr. 2, 2016. This is a diverse dataset of clear outdoornatural images and thus represents the type of scenes that might bedegraded by haze. We clustered the RGB pixel values of each image, suchby using K-means clustering, a Minimum Variance Quantization clustering,and/or the like, to a maximum of 500 clusters, and replaced every pixelin the image with its respective cluster center. The result is an imagewith 500 different RGB values at most (two orders of magnitude smallerthan image size). The PSNR (Peak Signal to Noise Ratio) of the imagesgenerated with the reduced color set, compared to the original ones,were high and ranged from 36.6 dB to 52.6 dB. A histogram of theobtained PSNR values is shown in FIG. 3 , as well as the image that hadthe worst PSNR, before and after color quantization. This validates thehaze-lines prior.

The observation regarding a small number of distinct colors holds forhaze-free images. In the presence of haze, object points that belong tothe same color cluster end up with different acquired colors since theyare located in disparate image areas and thus have different distancesfrom the camera. This prior suggests that pixels that are clusteredtogether in a haze-free image form a line in RGB space in a hazy image.Based on Eq. (1), the two end points of the line are the original colorJ and the airlight A. These are the haze-lines.

This prior is demonstrated in FIG. 1 . A haze-free image is clusteredusing K-means to 500 clusters. The pixels belonging to four of theseclusters are marked by different color markers in FIG. 1 a and their RGBcoordinates are plotted in FIG. 1 b , demonstrating tight clusters. Notethat the clusters include pixels distributed over the entire image thatcome from objects with different distances from the camera. A synthetichazy image was generated from the clear image (FIG. 1 c ) by the methodused in R. Fattal. Dehazing using color-lines. ACM Trans. Graph.,34(1):13, 2014. The same pixels as in FIG. 1 a are marked. However, now,colors of pixels that belonged to the same color cluster are no longersimilar. This is depicted in RGB space in FIG. 1 d , where the colorcoordinates of these pixels are distributed along a haze-line spanned bythe original color and the airlight. The pixels marked by purple circles(originating from the sand patch) are located in similar distances, sotheir distribution along the haze-line is rather tight. However, thepixels marked by orange triangles (grassy areas) are found at differentlocations in the real world, so they are distributed along thehaze-line.

FIG. 2 demonstrates the haze-lines prior on a hazy outdoor image. Sixdifferent pixels identified by our method as belonging to the same hazeline are circled. All of them are on shaded tree trunks and branches,and are likely to have similar radiance J. However, their observedintensity I is quite different, as shown in FIG. 2 b , where thesepixels form a haze-line in RGB space that passes through the airlight.

The present technique, in some embodiments thereof, is composed of threecore steps: clustering the pixels into haze-lines, estimating atransmission map, and dehazing. Optionally, the estimation of thetransmission map is divided into two: first, an estimation of an initialtransmission map; second, a regularization step which yields a moreaccurate transmission map.

Color Spaces and Multi/Hyper-Spectral Images

Embodiments of the present technique uses an example of an RGB colorchannel input image. When a non-RGB input image is received (such asCMYK, YIQ, YUV, YDbDr, YPbPr, YCbCr, xvYCC, HSV, HSL, etc.), it mayfirst be converted to RGB using techniques known in the art.Alternatively, the present technique may operate on any color space,with out without respective modifications. For example, the presenttechnique may work directly on non-RGB color spaces with lineartransformation to RGB space.

Optionally, equivalent embodiments may be applied to any spectral imagespace, such as two color channel, three color channel, four colorchannel, and/or the like. The maximum number of color channels that anembodiment may automatically process is limited by the limitations ofthe physical processing hardware, and may include fields of applicationsthat have other technical problems from those described herein, such asthe image dehazing of images depicting a landscape, seascape, and/or thelike. Therefore, the number of color channels of an image to beautomatically processed by an embodiment may be a range between 2 and15, 3 and 20, 4 and 10, 5 and 25, or any combination thereof.

Throughout this application, various embodiments of this invention maybe presented in a range format. It should be understood that thedescription in range format is merely for convenience and brevity andshould not be construed as an inflexible limitation on the scope of theinvention. Accordingly, the description of a range should be consideredto have specifically disclosed all the possible subranges as well asindividual numerical values within that range. For example, descriptionof a range such as from 1 to 6 should be considered to have specificallydisclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numberswithin that range, for example, 1, 2, 3, 4, 5, and 6. This appliesregardless of the breadth of the range.

Whenever a numerical range is indicated herein, it is meant to includeany cited numeral (fractional or integral) within the indicated range.The phrases “ranging/ranges between” a first indicate number and asecond indicate number and “ranging/ranges from” a first indicate number“to” a second indicate number are used herein interchangeably and aremeant to include the first and second indicated numbers and all thefractional and integral numerals therebetween.

Optionally, embodiments may be implemented for different imagingmodalities, such as different camera images, stereo camera images,photon sensor images, electromagnetic radiation images, particle images,and/or the like. The primary criterion for application to a modality isthat the “haze-lines” can be modelled in the color space as ananalytical shape (line, arc, parabola, etc.) and that the “airlightvalue can be used to remove the unwanted image characteristic.

Optionally, images may be in two dimensions, three dimensions, fourdimensions, five dimension, and/or the like. For example, a two channelembodiment may use the techniques described herein to partially processthe dehazing of an image, or otherwise remove unwanted imagecharacteristics (i.e. glare, prismatic effects, and/or the like). Forexample, multispectral or hyperspectral images may by processed, such asremote sensing atmospheric images comprising 5 color channels (i.e.atmospheric infrared transparency windows) to remove cloud cover,hazing, glare, and/or the like. Such augmented images may better be usedto compute sea surface temperature, vegetation indices, and/or the like.For example, dual-energy computed tomography images may be processedusing embodiments of the techniques to remove ghosting.

Specific Steps of Dehazing an Image

The first core step is finding the haze-lines. A may be estimated usingconventional methods, such those in R. Fattal. Single image dehazing.ACM Trans. Graph., 27(3):72, 2008; K. He, J. Sun, and X. Tang. Singleimage haze removal using dark channel prior. In Proc. IEEE CVPR, 2009;and R. Tan. Visibility in bad weather from a single image. In Proc. IEEECVPR, 2008.

Let us define I_(A) as:I _(A)(x)=I(x)−A,  Eq. (3)where the three-dimensional (3D) RGB coordinate system is translatedsuch that the airlight is at the origin. Following Eq. (1),I _(A)(x)=t(x)·[J(x)−A].  Eq. (4)

We express I_(A)(x) in spherical coordinates:I _(A)(x)=[r(x),θ(x),ϕ(x)].  Eq. (5)

Here r denotes the distance to the origin (i.e., ∥I−A∥)), θ and ϕ denotethe longitude and latitude, respectively.

The colors of the pixels are now represented in a spherical coordinatesystem around the airlight. FIG. 4 shows the histogram of the Forestimage (FIG. 2 a ) projected onto a sphere. The sphere was sampleduniformly using 500 points. The color at each point [ϕ, θ] indicates thenumber of pixels x with these angles when writing I_(A)(x) in sphericalcoordinates (image size 768×1024 pixels). The equator (ϕ=0) is marked bya bold dashed blue line, while the longitudes θ=0, π/2 are marked bydotted blue lines. The color-mapping is logarithmic for illustrationpurposes. The histogram indicates that the pixels are highlyconcentrated in terms of their longitude and latitude.

Let us look at Eq. (4). For given values of J and A, scene points atdifferent distances from the camera differ only in the value of t. Inthe spherical coordinate system we defined, changes in t affect onlyr(x) without changing either ϕ(x) or θ(x). In other words, pixels x andy have similar RGB values in the underlying haze-free image when their[ϕ, θ] are similar:

$\begin{matrix}{\left. {{J(x)} \approx {J(y)}}\Rightarrow\left\{ {{{\phi(x)} \approx {\phi(y)}},{{\theta(x)} \approx {\theta(y)}}} \right\} \right.,{\forall{t.}}} & {{Eq}.(6)}\end{matrix}$

Therefore, pixels belong to the same haze-line when their [ϕ(x), θ(x)]values are similar. Each point on the sphere in FIG. 4 represents ahaze-line, in which all the pixels have approximately the same angles[ϕ(x), θ(x)]. The pixels in each haze-line have similar values in thenon-hazy image J with high probability.

Note that there is inherent ambiguity between color and haze for colorswhich are collinear with the airlight:

$\begin{matrix}{{{J_{1} - A} = {\left. {\alpha\left( {J_{2} - A} \right)}\Rightarrow J_{1} \right. = {{\left( {1 - \alpha} \right)A} + {\alpha J_{2}}}}},} & {{Eq}.(7)}\end{matrix}$where α denotes a scale factor. In this case all single image dehazingmethods may correct J₁ and J₂ to the same color. This is the only casein our method when two color clusters may be mapped to the samehaze-line.

In order to determine which pixels are on the same haze-line, pixelsshould be grouped according to their angles [ϕ, θ]. A two-dimensional(2D) histogram binning of θ and ϕ with uniform edges in the range[0,2π]×[0, π] may not generate a uniform sampling of a sphere. Instead,the samples may be denser near the poles, as observed by G. Marsaglia.Choosing a point from the surface of a sphere. Ann. Math. Statist.,43(2):645-646, 04 1972, since the distance on the sphere is relative tosin(θ). Therefore, we sample the unit sphere uniformly, as shown in FIG.4 , where each vertex is a sample point. Each vertex corresponds to ahaze-line. For clarity of display, the number of samples in FIG. 1 issmaller than the actual number we use. We group the pixels based ontheir [ϕ(x), θ(x)] values, according to the closest sample point on thesurface to the airlight color channel values (i.e. the color differencesbetween the RGB values of the pixels of the digital image and theairlight color values). This may be implemented efficiently by buildinga KD-Tree from the pre-defined tessellation and querying the tree foreach pixel. This is much faster than running a clustering algorithm,such as K-means, Minimum Variance Quantization algorithm, or the like.

Based on the analysis of the prior described above, several hundreds ofhaze-lines represent an image with a good approximation. In someembodiment, the technique yields a range of between 10-50, 50-100,100-200, 200-300, 300-400, 400-500, 500-600, 600-700, 700-800, 800-900,or more than 900 haze-lines—each of these ranges constituting adifferent embodiment. In some embodiments, the number of haze-lines isdependent on the amount of colors in the image; generally, a verycolorful image would yield a large number of haze-lines (e.g., above400), while a relatively pale image would yield a lower number (e.g.,below 50). For example, in one experiment, an image of haystacks, whichincluded a relatively low number of colors, was well dehazed using aslittle as 20 haze-lines. FIG. 5 a depicts the layout of two differenthaze-lines in the image plane for the Forest image. Pixels belonging totwo different haze-lines are depicted in green and blue, respectively.FIG. 5 b is a histogram of r(x) within each cluster. The horizontal axisis limited to the range [0, ∥A∥], as no pixel may have a radius outsidethat range in this particular image.

The second core step of the present technique is to estimate thetransmission map. Optionally, this core step is broken into two. First,estimation of initial transmission: For a given haze-line defined by Jand A, r(x) depends on object distance:

$\begin{matrix}{{{r(x)} = {{t(x)}{{{J(x)} - A}}}},{0 \leq {t(x)} \leq 1.}} & {{Eq}.(8)}\end{matrix}$

Thus, t=1 corresponds to the largest radial coordinate:

$\begin{matrix}{r_{\max}\overset{def}{=}{{{J - A}}.}} & {{Eq}.(9)}\end{matrix}$

Combining Eqs. (8,9) results in an expression for the transmission basedon radii in the haze-line:t(x)=(x)/r _(max).  Eq. (10)

Now, the question is how to find an estimate {circumflex over (r)}_(max)for the maximal radius? When a haze-line H contains a haze-free pixel,then {circumflex over (r)}_(max) is the maximal radius of thathaze-line:

$\begin{matrix}{{{{\overset{\hat{}}{r}}_{m\alpha x}(x)} = {\max\limits_{x \in H}\left\{ {r(x)} \right\}}},} & {{Eq}.(11)}\end{matrix}$where the estimation is done per haze-line H. FIG. 5 b displays theradii histograms of the two clusters shown in FIG. 5 a . We assume thatthe farthest pixel from the airlight is haze free, and that such a pixelexists for every haze-line. This assumption does not hold for all of thehaze-lines in an image, however the regularization step partiallycompensates for it. Combining Eqs. (10,11) results in a per-pixelestimation of the transmission:{tilde over (t)}(x)=r(x)/{circumflex over (r)} _(max)(x).  Eq. (12)

Following the estimation of the initial transmission, a regularizationstep may take place due to the following reason. The initialtransmission is estimated using the haze-lines, without using anyspatial information. As a result, nearby pixels that were clustered todifferent haze-lines might have significantly different transmissionvalues, while in reality they are nearly at the same distance from thecamera. The regularization enforces the image smoothness on thetransmission. Where the image is smooth, we expect to find the sameobject at a similar distance and therefore expect the transmission tochange smoothly. On the other hand, when there is a significant gradient(color variance) in the image, it is likely to match to a depthdiscontinuity and we might see a discontinuity in the transmission aswell.

Since the radiance J is positive (i.e., J≥0), Eq. (1) gives a lowerbound LB on the transmission:

$\begin{matrix}{{t_{LB}(x)} = {1 - {\min\limits_{c \in {\{{R,G,B}\}}}\frac{I_{c}(x)}{A_{c}}}}} & {{Eq}.(13)}\end{matrix}$

In He at al. (described in Single image haze removal using dark channelprior, published in in Proc. of IEEE Conf Comput. Vis. Pattern Recognit.(CVPR) (June 2009), pp. 1956-1963, 978-1-4244-3991-1/09), thetransmission estimate is based on an eroded version of t_(LB). We imposethis bound on the estimated transmission, per-pixel:t _(LB)(x)=max{{tilde over (t)}(x),t _(LB)(x)}  Eq. (14)

The estimation in Eq. (12) is performed per-pixel, without imposingspatial coherency. This estimation may be inaccurate when a small amountof pixels were mapped to a particular haze-line, or in very hazy areas,where r(x) is very small and noise may affect the angles significantly.The transmission map should be smooth, except for depth discontinuities,as observed by Fattal et al. (2014), Id.; K. Nishino, L. Kratz, and S.Lombardi. Bayesian defogging. Int. Journal of Computer Vision (IJCV),98(3):263-278, 2012; Tan (2008), Visibility in bad weather from a singleimage, in Proc. IEEE CVPR, 2008; and J.-P. Tarel and N. Hautiere. Fastvisibility restoration from a single color or gray level image. InComputer Vision, 2009 IEEE 12th International Conference on, pages2201-2208, September 2009 (hereinafter Tarel).

We seek a transmission map {tilde over (t)}(x) that is similar tot_(LB)(x) and is smooth when the input image is smooth. Mathematically,we minimize the following function w.r.t. {tilde over (t)}(x):

$\begin{matrix}{{{\sum_{x}\frac{\left\lbrack {{\hat{t}(x)} - {{\overset{\sim}{t}}_{LB}(x)}} \right\rbrack^{2}}{\sigma^{2}(x)}} + {\lambda{\sum_{x}{\sum_{y \in N_{x}}\frac{\left\lbrack {{\hat{t}(x)} - {\hat{t}(y)}} \right\rbrack^{2}}{{{{I(x)} - {I(y)}}}^{2}}}}}},} & {{Eq}.(15)}\end{matrix}$where λ denotes a parameter that controls trade-off between the data andthe smoothness terms, N_(x) denotes the four nearest neighbors of x inthe image plane, and σ(x) denotes the standard deviation of {tilde over(t)}_(LB), which is calculated per haze-line.

σ(x) plays a significant role since it allows us to apply our estimateonly to pixels where the assumptions hold. When the variance is high,the initial estimation is less reliable. σ(x) increases as the number ofpixels in a haze line decreases. When the radii distribution in a givenhaze-line is small, our haze-line assumption does not hold since we donot observe pixels with different amounts of haze. In such cases, σ(x)increases as well.

The third core step of the technique is the dehazing: Once {circumflexover (t)}(x) is calculated as the minimum of Eq. (13), the dehazed imageis calculated using Eq. (1):

$\begin{matrix}{{\overset{\hat{}}{J}(x)} = {\left\{ {{I(x)} - {\left\lbrack {1 - {\overset{\hat{}}{t}(x)}} \right\rbrack A}} \right\}/{{\overset{\hat{}}{t}(x)}.}}} & {{Eq}.(16)}\end{matrix}$

The technique is summarized in Algorithm 1 below, and exemplary resultsthereof are demonstrated in FIG. 6 .

Intermediate and final results of our method: (a) the input hazy image,(b) the dehazed images, (c) the distance r(x) of every pixel in the hazyimage to the airlight, (d) the estimated radii {circumflex over(r)}_(max)(x) calculated according to Eq. (9). (e) The input image isshown, with the pixels x for which r(x)={circumflex over (r)}_(max)(x)marked by cyan circles, (f) The data term confidence in Eq. (13)colormapped (warm colors show the larger values), (g) the estimatedtransmission map {circumflex over (t)}(x) before the regularization, (h)the final transmission map {tilde over (t)}(x) after regularization. (g)and (h) are colormapped.

FIG. 6 a shows the input hazy image. The final, dehazed image is shownin FIG. 6 b . FIG. 6 c shows the distance r(x) in RGB space of everypixel in the hazy image to the airlight. Note that this distancedecreases as haze increases. FIG. 6 d shows the maximum radii{circumflex over (r)}_(max)(x) per haze-line, calculated according toEq. (11). Observe that FIG. 6 d is much brighter than FIG. 6 c . Sincelarger values are represented by brighter colors, this indicates thatthe distance to the airlight is increased. The pixels x with the maximumradius in their haze-line (for which for which r(x)={circumflex over(r)}_(max)(x)) are marked by cyan circles on the hazy image in FIG. 6 e. Note that these pixels are mostly at the foreground, where indeedthere a minimal amount of haze. We filtered out pixels that had amaximum radius in the haze line, yet had a σ>2, because the modelassumptions do not hold for these haze-lines. This happens in the sky,because the distance to the airlight in RGB space is so small thatclustering according to the angles is not reliable due to noise. In theregularization step this fact is taken into consideration through thedata-term weight 1/σ²(x), which is shown in FIG. 6 f , which iscolormapped data term confidence in Eq. (15) (warm colors depict highvalues). The ratio of FIGS. 6 c and 6 d yields the initial transmission{tilde over (t)}(x) that is shown in FIG. 6 g . The transmission mapafter regularization is shown in FIG. 6 h . While {tilde over (t)}(x)contains fine details even in grass areas that are at the same distancefrom the camera, {circumflex over (t)}(x) does not exhibit thisbehavior. This indicates the regularization is advantageous.

-   -   Algorithm 1: Haze Removal.    -   Input: I(x), A    -   Output: J(x), {circumflex over (t)}(x)        -   1: I_(A)(x)=I(x)−A        -   2: Convert I_(A) to spherical coordinates to obtain [r(x),            ϕ(x), θ(x)]        -   3: Cluster the pixels according to [ϕ(x), θ(x)].            -   Each cluster H is a haze-line.        -   4: for each cluster H do        -   5: Estimate maximum radius:            {circumflex over (r)} _(max)(x)=max_(x∈H) {r(x)}        -   6: for each pixel x do        -   7: Estimate transmission: {tilde over            (t)}(x)=r(x)/{circumflex over (t)}max        -   8: Perform regularization by calculating {circumflex over            (t)}(x) that minimizes Eq. 15        -   9: Calculate the dehazed image using Eq. (16)

Optionally, the clustering of pixels in spherical coordinates isperformed by representing the color differences between the pixels ofthe digital image and the airlight value of the pixels of the digitalimage on a pre-computed tessellation of the unit sphere, where thepre-computed tessellation is uniformly sampled and stored in Cartesiancoordinates in a k-dimensional tree (KD-tree). A KD-tree is acomputerized data structure for organizing points in a space with kdimensions. It is a binary search tree with constraints imposed on it.KD trees are very useful for nearest neighbor searches (i.e. in a colorspace). The searching for nearest neighbors on the KD-tree may beperformed using Euclidean distance coordinates. The pixel clusters aregrouped with the color samples based on the nearest neighbors, therebyproducing multiple groups each being one of the haze-lines.

As to the computational complexity of the present technique, thealgorithm is linear in N—the number of pixels in the image, andtherefore fast. The clustering is done using a nearest neighbor searchon a KD-Tree with a fixed number of points. Estimating the radius withineach cluster is linear in N. Therefore, the initial radius estimation isO(N). Seeking the minimum of Eq. (15) requires solving a sparse linearsystem, which is also O(N). Restoring the dehazed image from thetransmission map is O(N) as well.

Experimental Results of the Image Dehazing Technique

The inventors have evaluated the technique on a large dataset containingboth natural and synthetic images and compared its performance tostate-of-the-art algorithms. We assumed A is given, and used theairlight vector A calculated by M. Sulami, I. Geltzer, R. Fattal, and M.Werman. Automatic recovery of the atmospheric light in hazy images. InProc. IEEE ICCP, 2014 (hereinafter Sulami). We used the same parametersfor all of the images: in Eq. (15) we set λ=0.1 and we scaled 1/σ²(x) tobe in the range [0,1] in order to avoid numeric issues. In order to findthe haze-lines, we sampled uniformly 1000 points on the unit sphere(FIG. 4 shows only 500 for clarity).

A synthetic dataset of hazy images of natural scenes was introduced byFattal et al. (2014), Id., and is available online, athttp://www.cs.huji.ac.il/˜raananf/projects/dehaze_c1/results/, lastviewed Apr. 4, 2016. The dataset contains eleven haze free images,synthetic distance maps and corresponding simulated haze images. Anidentically-distributed zero-mean Gaussian noise with three differentnoise level: σ_(n)=0.01, 0.025, 0.05 was added to these images (withimage intensity scaled to [0,1]). Table 1 summarizes the L₁ errors onnon-sky pixels (same metric used in Fattal et al. (2014), Id.) of thetransmission maps and the dehazed images. Our technique is compared tothe method of Fattal et al. (2014), Id. and an implementation of He etal. (2009), Id. by Fattal et al. (2014), Id. For five images out of thisdataset, results of both clear and noisy images are provided by Fattalet al. (2014), Id.

Table 1: Comparison of L1 errors over synthetic hazy images with variousamount of noise. The noise standard deviation is given and the imagesare scaled to the range [0,1]. The table compares the L1 errors of theestimated transmission maps (left value) and the dehazed images (rightvalue).

The present σ He et al. (2009) Fattal et al. (2014) technique Road1 00.097/0.051 0.069/0.033 0.058/0.040 0.01 0.100/0.058 0.068/0.0380.061/0.045 0.025 0.106/0.074 0.084/0.065 0.072/0.064 0.05 0.136/0.1070.120/0.114 0.091/0.100 Lawn1 0 0.118/0.063 0.077/0.035 0.032/0.026 0.010.116/0.067 0.056/0.038 0.032/0.032 0.025 0.109/0.077 0.056/0.0650.052/0.056 0.05 0.115/0.102 0.114/0.121 0.099/0.107 Mansion 00.074/0.043 0.042/0.022 0.080/0.049 0.01 0.067/0.040 0.048/0.0300.088/0.056 0.025 0.057/0.044 0.065/0.051 0.104/0.072 0.05 0.083/0.0750.081/0.080 0.116/0.095 Church 0  0.07/0.048 0.039/0.025 0.047/0.0320.01 0.067/0.050 0.053/0.043 0.049/0.041 0.025 0.058/0.059 0.089/0.0810.047/0.057 0.05 0.087/0.121 0.121/0.136 0.043/0.092 Raindeer 00.127/0.068 0.066/0.034 0.089/0.045 0.01 0.119/0.066 0.077/0.0420.093/0.049 0.025 0.109/0.067 0.084/0.054 0.104/0.063 0.05 0.117/0.0850.106/0.083 0.131/0.092

As illustrated in Table 1, the present technique outperforms previousmethods in most cases, and handles the noise well. As expected, ourperformance degrades when the noise variance increases. However, ourtechnique maintains its ranking, with respect to other methods,regardless of the amount of noise. This shows that our algorithm isquite robust to noise, despite being pixel-based.

FIGS. 7 and 8 compare results to six state-of-the-art single imagedehazing methods: C. O. Ancuti and C. Ancuti. Single image dehazing bymultiscale fusion. IEEE Trans. on Image Processing, 22(8):3271-3282,2013 (hereinafter Ancuti); Fattal et al. (2014), Id.; K. B. Gibson andT. Q. Nguyen. An analysis of single image defogging methods using acolor ellipsoid framework. EURASIP Journal on Image and VideoProcessing, 2013(1), 2013; R. Luzon-Gonzalez, J. L. Nieves, and J.Romero. Recovering of weather degraded images based on RGB responseratio constancy. Appl. Opt., 2014; He et al. (2009), Id.; K. Nishino, L.Kratz, and S. Lombardi. Bayesian defogging. Int. Journal of ComputerVision (IJCV), 98(3):263-278, 2012; and K. Tang, J. Yang, and J. Wang.Investigating haze-relevant features in a learning framework for imagedehazing. In Proc. IEEE CVPR, 2014 (hereinafter Tang).

As previously noted by Fattal et al. (2014), Id., the image after hazeremoval might look dim, since the scene radiance is usually not asbright as the airlight. For display, we performed a global linearcontrast stretch on the output, clipping 0.5% of the pixel values bothin the shadows and in the highlights. Pixels whose radius is maximal intheir haze-line are marked in pink on the hazy input. We marked onlypixels x for which σ(x)<2 and for clarity, only ones that belong tolarge clusters.

The method of Ancuti et al. (2013), Id. leaves haze in the results, asseen in the areas circled in yellow. In the result of Luzon-Gonzalez etal. (2014), Id. there are artifacts in the boundary between segments(pointed by arrows). The method of Nishino et al. (2012), Id. tends tooversaturate (e.g., House). The methods of He et al. (2009), Id. andTang et al. (2014), Id. produce excellent results in general but lacksome micro-contrast when compared to Fattal et al. (2014), Id. and toours. This is evident in the zoomed-in buildings shown in Cityscaperesults, where in our result and in Fattal et al. (2014), Id. thewindows are sharper than in He et al. (2009), Id. and Tang et al.(2014), Id. The result of Gibson et al. (2013), Id. was not enlarged asit has a low resolution. Results of Fattal et al. (2014), Id. aresometimes clipped, e.g., the leaves in House and in the sky in Forest.Our assumption regarding having a haze-free pixel in each haze-line doesnot hold in Cityscape, as evident by several hazy pixels that set amaximum radius, e.g. the red buildings. Despite that, the transmissionin those areas is estimated correctly due to the regularization thatpropagates the depth information spatially from the other haze-lines.

FIG. 8 compares both the transmission maps and the dehazed images. Itshows our technique is comparable to other methods, and in certain casesworks better. For example, The two rows of trees are well separated inour result when compared to He et al. (2009), Id.

A major advantage of the global approach of the present technique is theability to cope well with fast variations in depth, when the details aresmaller than the patch size. FIG. 9 a shows an enlarged portion of animage, where clear artifacts are visible in the result of Fattal et al.(2014), Id. (FIG. 9 c ), around the leaves and at the boundary betweenthe trunk and the background. FIG. 9 d shows our result. A patch-basedmethod is less likely to estimate the distance of such scenesaccurately. The result of He et al. (2009), Id. does not exhibit theseartifacts in FIG. 9 b , because the dehazing is less effective in thisimage and the details are less clear (e.g., the circled trunk). Thisphenomena is also visible in FIG. 7 in the dehazed Cityscape image ofGibson et al. (2013), Id., where a halo between the trees in theforeground and the background is visible, and also in the train outputof Fattal et al. (2014), Id. around the pole (marked by a yellowsquare).

Using a fixed tessellation of the unit sphere might raise a concern thatfine tones may not be distinguished. FIG. 10 demonstrates this is notthe case. The pumpkins (a crop of FIG. 6 a ) are lit from above, andtherefore are brighter at the top and gradually become darker towardsthe ground (FIG. 10 left). FIG. 10 right depicts the cluster map—eachcolor symbolizes a different haze-line. The gradual tone change isevident in the cluster map.

The precise technique presented above in the framework of theexperimental results section is considered to be an embodiment of thepresent invention.

Airlight Calculation Using a Hough Transform

Optionally, a Hough transform in RGB space is used to automaticallycalculate airlight values, such as a set of color channel values for anairlight coordinate in RGB space. Hough transforms find imperfectinstances of haze-lines by a voting procedure, the voting procedurecarried out in a parameter space. Haze-line candidates are automaticallyobtained as local maxima in an “accumulator space” that is constructedby the Hough transform. For example, clusters of point are automaticallymodeled as haze-lines by the Hough transform, and each point in eachcluster may vote for the airlight RGB values, such as in a naïveembodiment.

Using the Hough transforms, a global airlight value may be automaticallydetermined in hazy images quickly and efficiently. The method is basedon the haze-line model introduced herein, that considers a cluster ofpixels intensities with similar colors to form lines in RGB space underhaze. These lines may intersect at the airlight color and we takeadvantage of this observation to find their point of intersection.

For example, given a candidate airlight coordinate in RGB space, wemodel pixels' intensities with a fixed set of lines emanating from theairlight candidate. That is, we wish to model pixels' values by anintersection point (i.e., the airlight) and a collection of lines (i.e.,the Haze-Lines). An airlight in the correct RGB location may fit thedata better than an airlight in a wrong location.

We search for an airlight point so that all lines emanating from theairlight point, in the given line directions, may fit the data. For thatwe use the Hough transform, where the point with the highest vote isassumed to be the airlight color. Running the Hough transform in threeor four dimensions (3D or 4D) may be computationally expensive, so wemay use two optional techniques to automatically accelerate thetechnique. One option is to work in 2D color spaces instead of a 3Dcolor space, for example, by automatically projecting pixels' values onthe RG, GB and RB planes. The second option is by automaticallyclustering pixels' values to collect votes for a candidate airlight fromcluster centers and weight each vote by a statistical parameter of thecluster, such as the cluster size, rather than collecting votes from allpixels. The actions for processing the digital images described hereinmay be performed completely automatically as no user intervention isrequired for the steps.

For example, we reduce the problem from 3D to 2D by considering theprojection of pixel values on the RG, GB and RB planes. We may combinethe votes in the three planes to obtain the final airlight estimation,such as a single airlight value selected from multiple airlightcandidates. As used herein the term airlight color channel value setmeans a set of three or more color channel values, corresponding to thepixel color channel values. For example, the set is a set of RGB values.This has a dramatic effect on the number of airlight candidates we needto sample and evaluate. Second, we may cluster all pixels in the imageinto roughly a thousand clusters. As a result of the optionalimprovements, the airlight value may be determined in a matter ofseconds, as opposed to minutes in the naïve implementation. Wedemonstrate our method on other real-world images and synthetic data.Our method may be more efficient than state-of-the-art methods (linearvs. quadratic complexity) and performs on-par with them. For example,the proposed algorithm's complexity is linear in the number of pixels inthe image, compared to alternatives which are quadratic. As a reference,the run-time of our MATLAB implementation on a desktop with a 4thgeneration Intel core i7 CPU @3.4 GHz and 32 GB of memory is on average6 seconds for a 1 Mpixel image.

Following is a detailed technical description of applying the Houghtransform technique for determining airlight values. When using a Houghtransform to estimate the airlight value, we may detect unknownparameters of a model given noisy data via a voting scheme. In thiscase, the voting procedure is carried out in a parameter spaceconsisting of candidate airlight values in RGB space. In particular, weuniformly sample a fixed set of line angles {θ_(k), ϕ_(k)}_(k=1) ^(K).Given this set, we consider a discrete set of possible airlight values.The distance between a pixel I(x) and the line defined by the airlightvalue A and a pair of angles (θ, ϕ) is:

$\begin{matrix}{{d\left( {{I(x)},\left( {A,\phi,\theta} \right)} \right)} = {{{\left( {A - {I(x)}} \right) \times \left( {{\cos(\theta)},\ {\sin(\phi)}} \right)}}.}} & {{Eq}.(35)}\end{matrix}$

A pixel vote may be assigned to a candidate A when the distance to oneof the lines is smaller than a threshold τ. This threshold is adaptiveand depends on the distance between A and I(x) to allow for smallintensity variations. For example, instead of working with cylinders(lines with a fixed threshold) we work with cones (lines with a variablethreshold). Formally:

$\begin{matrix}{\tau = {\tau_{0} \cdot {\left( {1 + \frac{{{I(x)} - A}}{\sqrt{3}}} \right).}}} & {{Eq}.(36)}\end{matrix}$

In addition, we allow a pixel to vote only for an airlight that isbrighter than the pixel, such as by computing the brightness from thecolor channel values and comparing. This is due to the fact that brightobjects are quite rare, as shown empirically to justify the dark channelprior, and usually do not contain information about the haze (e.g., abright building close to the camera).

The best representation of the pixels' values from a hazy image may befound with airlight A and fixed line directions {θ_(k), ϕ_(k)}_(k=1)^(K). This may be formulated as follows:

$\begin{matrix}{{\arg\max\limits_{A}{\sum_{x}{\sum_{k}{{1\left\lbrack {{d\left( {{I(x)},\left( {A,\phi_{k},\theta_{k}} \right)} \right)} < \tau} \right\rbrack} \cdot {1\left\lbrack {A > {I(x)}} \right\rbrack}}}}},} & {{Eq}.(37)}\end{matrix}$where 1[⋅] is an indicator function that equals 1 when true and equals 0otherwise. The term 1[A>I(x)] equals 1 when all elements of A aregreater than the corresponding elements of I(x).

A huge value of A>>1 might be chosen as the solution, since it maximizesEq. 37 the pixels in the same large cone. To prevent this, we give alarger weight to values of A that are close to the pixel's values.Formally, we optimize:

$\begin{matrix}{{\left. {\left. {\arg\max\limits_{A}{\sum_{x}{\sum_{k}{{f\left( {{{I(x)} - A}} \right)} \cdot {1\left\lbrack {d\left( {{I(x)},A,\phi_{k},\theta_{k}} \right)} \right.}}}}} \right) < \tau} \right\rbrack \cdot {1\left\lbrack {A > {I(x)}} \right\rbrack}},} & {{Eq}.(38)}\end{matrix}$where f(y)=1+4·e^(−y) is a fast decaying weight that gives preference tovalues of A in the vicinity of the pixel's distributions.

Optimizing Airlight Computational Efficiency

The proposed scheme, which includes collecting votes from all pixels forall angles and airlight candidates in the 3D RGB space, iscomputationally expensive. Therefore, we propose the followingapproximations, which significantly accelerate the computation whilemaintaining accuracy. The first, clustering the colors in the image andusing the cluster centers instead of all the pixels. The second,performing the voting scheme in two dimensions. The voting is repeatedthree times, with only two of the (R, G, B) color channels being usedeach time.

Color clusters may be quantized before the Hough voting, such asquantizing the image into N clusters. We may do this by converting theRGB image into an indexed image with a unique color palette of length N.This may give us a set of N typical color values, {I_(n)}_(n=1) ^(N),where N is much smaller than the number of pixels in the image. Inaddition, we have {w_(n)}_(n=1) ^(N), the number of pixels in the imagebelonging to each cluster. During the Hough voting procedure, eachrepresentative color value I_(n) votes based on its distance to thecandidate airlight, and the vote has a relative strength w_(n).Therefore, the final optimization function is:

$\begin{matrix}{\arg\max\limits_{A}{\sum\limits_{n}{\sum\limits_{k}{w_{n} \cdot {f\left( {{I_{n} - A}} \right)} \cdot {1\left\lbrack {{d\left( {I_{n},\left( {A,\phi_{k},\theta_{k}} \right)} \right)} < \tau} \right\rbrack} \cdot {{1\left\lbrack {A > I_{n}} \right\rbrack}.}}}}} & {{Eq}.(39)}\end{matrix}$

Calculating the full 3D accumulator for all possible airlight values iscomputationally expensive. Therefore, voting e may be done in a lowerdimension. The accumulator may be seen as the joint probabilitydistribution of the airlight in all color channels, where the finalselected value is the one with the maximal probability. By performingthe accumulation two color channels at a time, we calculate threemarginal probabilities, where each time the summation is performed on adifferent color channel. Finally, we look for a candidate airlight thatmay maximize the 3D volume created by the outer product of the marginalaccumulators. The proposed Hough technique is summarized in Algorithm 2.

-   -   Algorithm 2: Airlight Estimation    -   Input: hazy image, I(x)    -   Output: airlight value, Â    -   1: cluster the pixels' colors and generate an indexed image Î(x)        whose values are n∈{1, . . . , N}, a colormap {I_(n)}_(n=1)        ^(N), and cluster sizes {w_(n)}_(n=1) ^(N)    -   2: for each pair of color channels (c₁, c₂)∈{RG, GB, RB} do    -   3: initialize accum_(c) _(1,) _(c) ₂ to zero    -   4: for A=(m·ΔA,l·ΔA), m, l∈{0, . . . , M} do    -   5: for θ_(k)=π_(K), k∈{1, . . . , K} do    -   6: for n∈{1, . . . , N} do    -   7: d=|(A−I_(n)(c₁,c₂))×(cos(θ_(k)),sin (θ_(k)))|    -   8: if (d<τ)∧(m·ΔA>I_(n)(c₁))∧(l·ΔA>I_(n)(c₂)) then        -   accum_(c) _(1,) _(c) ₂ (k,m,l)+=w_(n)·f(∥A−I_(n)∥)    -   9: Â=argmax{accum_(R,G)⊗accum_(G,B)⊗accum_(R,B)}, where ⊗ is an        outer product    -   10: Return

The algorithm's run-time depends on the following parameters: the numberof pixels in the image P, the number of airlight candidates (in eachcolor channel) M, the number of color clusters N and the number ofhaze-line orientations K. The conversion from RGB to an indexed imagehas a run-time complexity of O(NP), while the airlight estimation usingthe indexed image has a run-time complexity of O(NKM²).

Reference is now made to FIG. 18 , which shows an exemplary selection ofairlight values 18A, 18B, and 18C from Hough transform. Hough transformvotes for the image show color clusters {I_(n)}_(n=1) ^(N) projectedonto 3 different 2D planes 18D, 18E, and 18F. Each cluster n is markedby a circle with a size proportional to w_(n). The ground-truth (GT)airlight is marked by a green circle while our estimate is marked by apurple diamond. The GT values are determined by a train operatorselecting the airline values manually from the image. Each coloredcluster votes for the GT value, where different colors indicatedifferent haze-lines. The gray colored clusters do not vote for the GTsince the following holds: 1[A>I_(n)]=0. The three voting arrays 18A,18B, and 18C show accum_(c) _(1,) _(c) ₂ , (c₁, c₂)∈RG, GB, RB as afunction of the candidate air-light values. The color-map indicates thenumber of votes, and an airlight value set is selected from theaccumulated assigned votes from all pixels. In this case, theground-truth air-light had the most votes in all planes (strong yellowcolor). The bottom row images 18D, 18E, and 18F show the distribution ofthe clustered pixels in RGB space. We show 2D plots since theseprojections are used in the 2D voting procedure (step 8 of Algorithm 1)and may improve visualization. The airlight value is pointed at by thestrongest haze-lines.

Experimental Results of the Hough Airlight Technique

We validate the proposed method on a diverse set of images. In all ofour experiments we use the following parameters: N=1000, the number ofcolor clusters for each image (some images have less typical colors,resulting in empty clusters and N<1000 in practice); K=40, the number ofangles, i.e., haze-lines, in each plane; all of the pixels' intensitiesare normalized to the range [0,1], and therefore we set ΔA=0.02 and1/ΔA; the threshold to τ=0.02 determines whether a pixel I, supports acertain haze-line.

Reference is now made to FIG. 19 , which shows images used forcomparison of airlight values computed with Hough transform and withalternative techniques. These images show the results from evaluatingthe accuracy of the estimated airlight on natural images with differenttechniques. The results are from the techniques of Sulami, He et al,Bahat et al (described in Blind dehazing using internal patchrecurrence, Proceedings of the 2016 IEEE International Conference onComputational Photography (ICCP), 13-15 May 2016, EISBN:978-1-4673-8623-4, DOI: 10.1109/ICCPHOT.2016.7492870), and the presentinvention. Following is table 2 summarizing the L₂ errors between thetechniques from analysis of 40 images.

TABLE 2 Statistic Ours Bahat He Sulami Mean L2 error 0.119 0.111 0.1290.255 Median L2 error 0.063 0.086 0.109 0.191 Variance 0.014 0.013 0.0130.061

Generally, the present embodiment and Bahat outperformed the others.Compared to Bahat, our embodiment results in a lower median error, withslightly higher mean and variance. The performance depends on the extentthe image adheres to the prior used by each method.

Images 19A, 19B, 19C, and 19D show examples of hazy images, along withtheir manually extracted ground-truth airlight (GT) colors (modified toin gray scale). Following are tables of values corresponding to theairlight colors of the images by the different methods.

TABLE 3 range: 0-1 Image 19A name = road.png He 0.94118 0.97647 0.98824Sulami: 1.1842 1.2987 1.353 Bahat: 0.85481 0.92456 0.94885 ours: 0.960.99 1 GT: 0.84428 0.93775 0.97859 Image 19B name = schechner.png He:0.76078 0.73333 0.6902 Sulami: 0.48603 0.8293 1.1188 Bahat: 0.847180.79573 0.73942 ours: 0.52 0.63 0.8 GT: 0.50074 0.59792 0.75972 Image19C name = train.png He: 0.70588 0.70588 0.71373 Sulami: 510183.8510183.5 510185 Bahat: 0.75459 0.75287 0.7529 ours: 0.72 0.71 0.72 GT:0.72548 0.72693 0.73123 Image 19D name = underwaterVessel.png He:0.56471 0.71765 0.68235 Sulami: 0.16142 2.2662 3.5469 Bahat: 0.421210.76008 0.93351 ours: 0.42 0.75 0.92 GT: 0.28975 0.60814 0.83459

TABLE 4 range: 0-255 Image 19A name = road.png He: 240 249 252 Sulami:255 255 255 Bahat: 218 236 242 ours: 245 252 255 GT: 215 239 250 Image19B name = schechner.png He: 194 187 176 Sulami: 124 211 255 Bahat: 216203 189 ours: 133 161 204 GT: 128 152 194 Image 19C name = train.png He:180 180 182 Sulami: 255 255 255 Bahat: 192 192 192 ours: 184 181 184 GT:185 185 186 Image 19D name = underwaterVessel.png He: 144 183 174Sulami: 41 255 255 Bahat: 107 194 238 ours: 107 191 235 GT: 74 155 213

The error bars corresponding to them in 19E are labeled. In the Roadimage 19A our error is larger than Bahat. This may be caused by severalbright pixels that have a high red value. In the Schechner image 19B ourmethod outperforms all methods. In the Train image 19C shows that allmethods except Sulami perform well. In the Vessel image 19D all methodsyield relatively high errors. This may be because the airlight is notuniform across the scene.

Reference is now made to FIG. 20 , which shows a comparison ofselections of airlight values from Hough transform for images 20A, 20B,20C, 20D, 20E, 20F, 20G, and 20H with and without visible sky. First, weestimated the airlight using the entire image, and received an averageerror of 0.116 (median error 0.091). Second, in order to testperformance in images that do not contain sky patches, we repeated theestimation process with a cropped image. The average error increased to0.231 (median error 0.25). In FIG. 20 the images the cropped region ismarked by a dotted line. The estimated airlight values of the full andcropped images are shown, as well as the GT value extracted manuallyfrom the images. Our cropped image estimations are close to the onesestimated from the full image. The largest error, both before and aftercropping, was calculated for the right image on the second row from thetop—it had an L₂ error of 0.35.

Reference is now made to FIG. 21 , which shows a comparison of dehazingbetween Hough transform 21C and 21D and an alternative technique 21A and21B. Shown are end-to-end dehazing results using both the airlightestimation described in He et al and the airlight estimation of anembodiment of the current technique. Both using an embodiment and thedehazing method described herein. Using different airlight values showsthe effect of the estimation on the output dehazed image. Images 21C and21D show a successful example where the airlight was accuratelyestimated for the Schechner image 19B. The incorrect value estimated byHe et al leads to an incorrect transmission map 21B, while thetransmission in 21D approximately describes the scene structure. In thetransmission maps, the darker colors are farther from the sensor, andthe lighter are closer. Seen in 21B, the previous technique, the imageintensities cause the brighter buildings to be considered farther(bottom right), awhile in 21D the transmission maps shows much bettervisual correspondence with the “depth” of the depicted objects.

Following are results of a comparative analysis of synthetic images. InSulami, the images were simulated from haze-free RGB images and theirdistance maps, gathered from the Lightfields and the Middlebury datasetsused respectively in He et al and Scharstein et al (described in Ataxonomy and evaluation of dense two-frame stereo correspondencealgorithms, IJCV, 47(1-3):7-42, 2002). The transmission maps werecalculated by t(x)=e^(−βd(x)), and β was chosen such that the mostdistant object in the scene received t=0.1. The airlight magnitude wasuniformly sampled in the range [0.8,1.8] and the orientation wasuniformly sampled from the 10° cone around [1,1,1]. The sampling processwas repeated three times for each image and the results are reported inSulami. We did not perform a per-image comparison of the techniques.Instead we report average and median errors in Table 5.

TABLE 5 Orientation He Tan Tarel Sulami Ours Mean 3.218 3.576  3.253 0.581 0.043 Median 3.318 3.316  3.49  0.22 0.037 Magnitude He Tan TarelSulami Ours Mean 0.172 0.218 0.412 0.157 0.178 Median 0.141 0.208 0.3930.116 0.095 l_(∞) Endpoint Error He Tan Tarel Sulami Ours Mean 0.1470.177 0.278 0.103 0.141 Median 0.144 0.178 0.286 0.077 0.106

Some of the images in this dataset are indoor images, whose depthdistribution is significantly different from that of outdoor images.Despite that, our results are competitive. Specifically, our orientationestimation is the most accurate, which is significant. It has been shownin [15] that estimating the airlight's orientation is more importantthan its magnitude, since errors in the orientation induce colordistortions in the dehazed image, whereas magnitude errors induce onlybrightness distortions. FIG. 5 shows two examples of synthetic imagesused in this experiment.

Underwater Image Restoration

As light propagates in water it is attenuated and scattered. Botheffects depend on the distance the light travels and its wavelength, asshown, for example, by Mobley, C. D.: Light and water: radiativetransfer in natural waters. Academic press (1994). Thewavelength-dependent attenuation causes color distortions that depend onthe object's distance and therefore cannot be globally compensated for.The scattering induces a distance dependent additive component on thescene that reduces contrast. As a result, many underwater images appearblue and lack vivid colors.

Nevertheless, color and contrast are extremely important for visualsurveys in the ocean. For example, enhanced images may improve automaticsegmentation, increase the quality of feature matching between imagestaken from multiple viewpoints, and aid in identification.

Present embodiments aim to recover the object's colors in scenesphotographed under ambient illumination in water using solely a singleimage as an input. Another aim is to recover a distance map of thephotographed scene. This problem is closely related to the single imagedehazing problem discussed above, in which images are degraded byweather conditions such as haze or fog. The above dehazing techniqueassumes that the attenuation is uniform across colors.

Under water, where the assumption of color-independent attenuation doesnot hold, there are theoretically three unknown transmission values perpixel (one per channel), yielding six unknowns with only threemeasurements. However, the color-dependent transmission is related tothe distance via the attenuation coefficients. Based on this relation weshow that the problem may be reduced to four unknowns per pixel asbefore, with two new global parameters—the ratios between theattenuation coefficients of the color channels.

We show that when the attenuation ratios between the different channelsare known, then the input image may be converted to a medium-compensatedimage where the attenuation coefficient is the same for all colorchannels. Then, the above image dehazing technique may be used to solvethe problem. In alternative embodiments, other image dehazing techniquesmay be applied to the medium-compensated image. We are left with thequestion of how to estimate the two additional global parameters. Tothis end, we show that using the wrong parameters results in images withdistorted colors. Hence, we automatically choose the parameters as theones that yield the best looking image. This is defined as the imagethat best adheres to the gray world assumption, that was used before forabove and under water imaging. We find the correct parameters bysampling the parameter space, which is bounded by known physicalmeasurements of naturally occurring water types.

The results of experiments conducted by the inventors demonstrate singleimage restoration of underwater scenes using the full physical imageformation model. Thus, we are able to recover complex 3D scenes and, inaddition, estimate the water properties.

We follow the model developed in Schechner, Y. Y., Karpel, N.: Recoveryof underwater visibility and structure by polarization analysis. IEEE J.Oceanic Engineering 30(3) (2005) 570-587. In each color channel c∈{R, G,B}, the image intensity at each pixel is composed of two components,attenuated signal and veiling-light:

$\begin{matrix}{{{I_{c}(x)} = {{{t_{c}(x)}{J_{c}(x)}} + {\left( {1 - {t_{c}(x)}} \right) \cdot A_{c}}}},} & {{Eq}.(17)}\end{matrix}$where x denotes the pixel coordinate, I_(c) denotes the acquired imagevalue in color channel c, t_(c) denotes the transmission of that colorchannel, and Je denotes the image value of the object that would havebeen acquired without the scattering and absorption of the water medium.The global veiling-light component A_(c) denotes the scene value inareas with no objects (t=0). Eq. (17) applies to linear captured data,prior to in-camera processing such as color-space conversion, gammacorrection and compression. Therefore, I refers to the image obtainedfrom the raw file after minimal processing such as demosaicing and blackcurrent subtraction, as disclosed by Akkaynak, D., Treibitz, T., Xiao,B., Girkan, U. A., Allen, J. J., Demirci, U., Hanlon, R. T.: Use ofcommercial off-the-shelf digital cameras for scientific data acquisitionand scene-specific color calibration. JOSA A 31(2) (2014) 312-321; andin Sumner, R.: Processing raw images in matlab.https://users.soe.ucsc.edu/rcsumner/rawguide/RAWguide.pdf (2014), lastviewed Apr. 4, 2016.

The transmission depends on object distance z and the water attenuationcoefficient for each channel β_(c):t _(c)=exp(−β_(c) z).  Eq. (18)

Under water, the attenuation of red colors may be an order of magnitudelarger than the attenuation of blue and green, os observed, for example,by Mobley, C. D.: Light and water: radiative transfer in natural waters.Academic press (1994). Therefore, as opposed to the common assumption insingle image dehazing, the transmission t is wavelength-dependent.

Jerlov, N. G.: Marine optics. Volume 14. Elsevier (1976) developed afrequently used classification scheme for oceanic waters, based on waterclarity. The Jerlov water types are I, IA, IB, II and III for open oceanwaters, and 1 through 9 for coastal waters. Type I is the clearest andtype III is the most turbid open ocean water. Likewise, for coastalwaters, type 1 is clearest and type 9 is most turbid. FIG. 11 (Left)depicts the attenuation coefficients of Jerlov water types; the figurewas adapted from data in Mobley et al. (1994), Id., based onmeasurements in Austin, R., Petzold, T.: Spectral dependence of thediffuse attenuation coefficient of light in ocean waters. OpticalEngineering 25(3) (1986) 253471-253471. Solid lines mark open oceanwaters while dashed lines mark coastal waters.

When capturing an image using a commercial camera, three color channelsR, G, B are obtained. Thus, we are interested in three attenuationcoefficients: (β_(R), β_(G), β_(B)). We show below that the threeattenuation coefficient themselves are not required for transmissionestimation, but rather their ratios (two variables). FIG. 1 (Right)shows the ratios of the attenuation coefficients: β_(B)/β_(H) vs.β_(B)/β_(G) of Jerlov water types for wavelengths of peak camerasensitivity according to Jiang, J., Liu, D., Gu, J., Susstrunk, S.: Whatis the space of spectral sensitivity functions for digital colorcameras? In: Proc. IEEE Workshop Applications of Computer Vision (WACV).(2013) 168-179.

FIG. 12 demonstrates the effect of scattering medium on observed scenecolors.

FIG. 12 a shows a scene with four different clusters of similar colors Jmarked. Pixels of a haze-free color image are clustered using K-means.Pixels belonging to four of the clusters are marked. Note that thepixels are non-local and are spread all over the image plane.

FIG. 12 b shows the same pixels in RGB space, with colors of theclusters corresponding to the highlighted pixels in FIG. 12 b.

FIG. 12 c shows the same scene with added synthetic haze. The sameclustered pixels are marked, but their observed colors are affected bydifferent amounts of haze.

FIG. 12 e shows the scene as if it was captured under water.

FIGS. 12 d and 12 f show the corresponding pixels in RGB space for hazeand underwater, respectively. In FIG. 12 d , the hazy pixels aredistributed haze-lines passing through the airlight, marked in black. InFIG. 12 f , the cluster pixels are distributed along curves that do notcoincide with the linear lines spanned by the original color and theveiling-light (the haze-lines), due to the wavelength-dependentattenuation. In haze, the attenuation is mostly wavelength independentand the tight color clusters become haze-lines. However, under water,the attenuation is wavelength-dependent, and the clusters become curves.

We first show that the absolute values of the attenuation coefficientsare not required for recovery. Instead, we show how to reconstruct thescene using only two global ratios between the attenuation coefficients.Then, we show how to estimate the ratios of the attenuation coefficientsfrom the image itself.

We modify the non-local single image dehazing technique discussed above,to take into account different attenuation coefficients for thedifferent color channels.

Given attenuation ratios, we convert the input image into a mediumcompensated image where all three channels have the same attenuationcoefficient. Then we apply the present dehazing technique (or adifferent one known in the art) to solve the problem.

We assume A_(c) is extracted from a patch in the image.

Combining and rearranging Eqs. (1a, 2a) yields for the blue channel:

$\begin{matrix}{{{A_{B} - I_{B}} = {e^{{- \beta_{B}}Z} \cdot \left( {A_{B} - J_{B}} \right)}},} & {{Eq}.(19)}\end{matrix}$and the same for the red channel:

$\begin{matrix}{{A_{R} - I_{R}} = {e^{{- \beta_{R}}Z} \cdot {\left( {A_{R} - J_{R}} \right).}}} & {{Eq}.(20)}\end{matrix}$

Raising Eq. (20) to the power of yields

$\begin{matrix}{\left( {A_{R} - I_{R}} \right)^{\frac{\beta_{B}}{\beta_{R}}} = {{e^{{- \beta_{R}}{z \cdot \frac{\beta_{B}}{\beta_{R}}}} \cdot \left( {A_{R} - J_{R}} \right)^{\frac{\beta_{B}}{\beta_{R}}}} = {t_{B} \cdot {\left( {A_{R} - J_{R}} \right)^{\frac{\beta_{B}}{\beta_{R}}}.}}}} & {{Eq}.(21)}\end{matrix}$

Denote the ratios between the attenuation coefficients:β_(BR)=β_(B)/β_(R),β_(BG)=β_(B)/β_(G).  Eq. (22)

Then, in this medium-compensated space we achieve a form similar to Eq.(1), with one unknown transmission per-pixel, common to all colorchannels:

$\begin{matrix}{\begin{bmatrix}\left( {{I_{R}(x)} - A_{R}} \right)^{\beta_{BR}} \\\left( {{I_{G}(x)} - A_{G}} \right)^{\beta_{BG}} \\\left( {{I_{B}(x)} - A_{B}} \right)\end{bmatrix} = {{t_{B}(x)}\begin{bmatrix}\left( {{J_{R}(x)} - A_{R}} \right)^{\beta_{BR}} \\\left( {{J_{G}(x)} - A_{G}} \right)^{\beta_{BG}} \\\left( {{J_{B}(x)} - A_{B}} \right)\end{bmatrix}}} & {{Eq}.(23)}\end{matrix}$

This form is similar to the haze-lines formulation. We expect to findhaze-lines in the medium-compensated space, where the transmission ofthe blue channel spans the haze-lines.

Scene Recovery: Once t_(B) is estimated, we may compensate for the colorattenuation using the following:

$\begin{matrix}{J_{c}{= {{A_{c} + \frac{I_{c} - A_{c}}{e^{{- \beta_{c}}Z}}} = {A_{c} + \frac{I_{c} - A_{c}}{t_{B}^{\frac{\beta_{c}}{\beta_{B}}}}}}}} & {{Eq}.(24)}\end{matrix}$where c∈{R, G, B}.

Eq. (24) compensates for the intensity changes that happen in the pathbetween the object and the camera. In addition, the ambient illuminationis attenuated by the water column from the surface to the imaging depth,resulting in a colored (bluish) global illumination. We are interestedin restoring the colors as when they were viewed under white light.Since this effect is global in the scene, we correct it by performing aglobal white balance on the result. This global white balance works wellonly because the distance-dependent attenuation and scattering effectshave already been compensated for. Otherwise, as demonstrated in FIG. 14, it has little effect. That is, the contrast in the image may not beuniform and further objects may have a reduced contrast.

Finally, since Eq. (17) applies to the linear captured data, we convertthe linear image to sRGB using a standard image processing pipeline,including color-space conversion from the sensor-specific to a standardsRGB, and a gamma curve as in Sumner, R.: Processing raw images inmatlab. https://users.soe.ucsc.edu/rcsumner/rawguide/RAWguide.pdf(2014), last viewed Apr. 4, 2016.

We have shown that accounting for color-dependent attenuation requiresonly two additional global parameters. Next we show how to estimate themautomatically.

Using the wrong coefficients results in reconstructions that are colorskewed. We use this insight to search for the most appropriate watertype. We perform the restoration multiple times using differentattenuation coefficients corresponding to different water types, andchoose the best result automatically based on a variant of the grayworld assumption.

FIG. 11 (right) shows the approximate attenuation coefficient ratios#_(BG), β_(BR), calculated for different Jerlov water types (FIG. 11 a). For each color channel, we take the attenuation at the peakwavelength of typical camera sensitivity responses based on Jiang et al.(2011), Id.: 475 nm, 525 nm and 600 nm for B, G, R, respectively. Thesevalues are a mere approximation, since they are based on a singlewavelength, while cameras have a wideband response. However, ouranalysis shows this camera-independent approximation works well inpractice: taking into account a wideband response did not yield anoticeable difference in the restoration.

According to the Gray-World assumption of Lu, H., Li, Y., Serikawa, S.,Underwater image enhancement using guided trigonometric bilateral filterand fast automatic color correction. In: Image Processing (ICIP), 201320th IEEE International Conference on. (September 2013) 3412-3416, theaverage reflectance of surfaces in the world is achromatic. It has beenused in the past for estimating attenuation coefficients underwaterusing known distances, such as by Bryson, M., Johnson-Roberson, M.,Pizarro, O., Williams, S. B.: Colour-consistent structure-from-motionmodels using underwater imagery. In: Robotics: Science and Systems,Citeseer (2012) 1-8. A significant portion of images taken under wateroften contains water without any objects. The Gray-World assumptionobviously does not hold there. Therefore, we apply the Gray-Worldassumption only at image regions that contain objects, i.e., those thatwere not identified as veiling-light pixels. Thus, among all results fordifferent water types, we choose the image where the difference betweenthe average values of the red, green, and blue channels is the smallest.

We considered several other measures such as maximal contrast (such asin Tan (2008)), Gray-World assumption on all three color channels andthe maximal eigen-value of the RGB-histogram (looking for a similarcolor distribution among channels). We found that a simple Gray worldassumption on non-veiling pixels gave the best results and therefore wefocus on this measure.

The present restoration technique is summarized in Algorithm 3 below.

-   -   Algorithm 3: Underwater image restoration.    -   Input: I(x), A    -   Output: Ĵ(x), {circumflex over (t)}(x)        -   1: for each (β_(BR),γ_(BG)) values of water types (FIG. 10            right) do        -   2: for each c∈{R, G, B} do        -   3:

${{\overset{\sim}{I}}_{c}(x)} = {{sign}{\left( {{I_{c}(x)} - A_{c}} \right) \cdot {{abs}\left( {{I_{c}(x)} - A_{c}} \right)}^{\beta_{Bc}}}}$

-   -   -   -   (see definition of β_(BC) in Eq. (22))

        -   4: Convert Ĩ_(c) to spherical coordinates to obtain [r(x),            ϕ(x), θ(x)]

        -   5: Cluster the pixels according to [ϕ(x), θ(x)]            -   Each cluster H is a haze-line.

        -   6: for each cluster H do

        -   7: Estimate maximum radius: {circumflex over            (r)}_(max)(x)=max_(x∈H){r(x)}

        -   8: for each pixel x do

        -   9: Estimate transmission:

${{\overset{˜}{t}}_{B}(x)} = \frac{r(x)}{{\hat{r}}_{\max}}$

-   -   -   10: Perform regularization by calculating {circumflex over            (t)}(x) that minimizes Eq. (15)        -   11: Calculate the restored image using Eq. (24)        -   12: Perform a global WB on the restored image        -   13: Calculate the mean colors of non-veiling-light pixels        -   14: Return the image with the mean colors closest to gray

Optionally, we choose and return the image that best conforms to theGray-World assumption, on non-veiling-light pixels. Optionally, othermethods for computing a parameter using the image data, and choosing oneof the images based on the parameter. For example, parameters may bestandard deviations of one or more color channel values, otherstatistical values of the RGB color values, and/or the like.

Experimental Results of the Underwater Image Restoration Technique

We first discuss implementation details of the underwater haze-linesvariation. According to Eq. (23), we expect to find haze-lines in themedium-compensated space:

$\begin{matrix}\left\lbrack {\left( {{I_{R}(x)} - A_{R}} \right)^{\beta_{BR}},\left( {{I_{G}(x)} - A_{G}} \right)^{\beta_{BG}},\left( {{I_{B}(x)} - A_{B}} \right)} \right\rbrack & {{Eq}.(25)}\end{matrix}$

The ratios denoted β_(BR) and β_(BG) are often fractions, and theambient light denoted A may be larger than the acquired color I. Inorder to avoid numerical problems, we calculate the colors in themedium-compensated space as follows:

$\begin{matrix}\begin{bmatrix}{{{sign}\left( {{I_{R}(x)} - A_{R}} \right)} \cdot {{abs}\left( {{I_{R}(x)} - A_{R}} \right)}^{\beta_{BR}}} \\{{sign}{\left( {{I_{G}(x)} - A_{G}} \right) \cdot {{abs}\left( {{I_{G}(x)} - A_{G}} \right)}^{\beta_{BG}}}} \\\left( {{I_{B}(x)} - A_{B}} \right)\end{bmatrix} & {{Eq}.(26)}\end{matrix}$

We then cluster the points in the medium-compensated space intohaze-lines. Due to the smaller variety of colors in the underwaterenvironment, which stems partially from the narrower spectrum ofillumination, we use only 500 points sampled uniformly on a sphere, incontrast to the 1000 sampled points in the experiments of the dehazingtechniques discussed above.

Once the haze-lines are obtained, we calculate the transmission of eachpixel according to the ratio between its distance to the veiling-lightand the distance of the most distant pixel in that haze-line. While inair it is somewhat reasonable to assume there is an almost haze-freepixel in each haze-line, under water this assumption does not hold. Evenscene points that are located at a distance of one meter from the camerahave a transmission of about 0.9 in the blue channel, depending on watertype. Therefore, we multiply the initial transmission estimation by 0.9even before the regularization.

We found the underwater data to be noisier then haze images. Therefore,we set 1/σ(x²) in the regularization term to be 1 when the haze line hasmore than 50 pixels and when the radius of the pixels at i is largerthan 0.1.

We used raw images taken with a Canon 5DII and a Nikon D810 in threedifferent locations, in tropical waters and in murkier coastal water.Two different color charts were placed in the scenes for verification:one, based on the X-rite color checker (by X-Rite Inc., Michigan, USA),and the second is QPcard-202 (by QPcard AB, Sweden), both encased forwater protection, with matte coating.

The color charts are used only for validation. During the transmissionestimation, we masked out the color charts, in order to estimate thetransmission based on natural scene objects alone. The transmission ofthose pixels is determined during the regularization step based onneighboring values.

FIG. 13 demonstrates the importance of choosing the correct water type.Using an incorrect value leads to an incorrect transmission estimation.As a result, some area in the image may be under- or over-compensated inthe restoration process. FIG. 13 a shows an input image with X-ritecolor checker, and two different outputs (FIGS. 13 b and 13 c ) for twowater types, including a zoom-in on the color-checker. The color-checkeris used only for validation of the restoration. Direct comparison ofcolor-checker values requires that both images are captured under thesame global illumination conditions. Since the global illumination inour case is unknown, we measure the angle θ in RGB space between thecolor of the gray-level patches in our result and the direction of avector with equal color components [1,1,1]. In a perfect restoration θ=0and cos(θ)=1. The correct water-type achieves a value of cos(θ)=0.99while the incorrect achieves a value of cos(θ)=0.96.

In FIG. 13 a , an image is restored using two different water types(different β_(BR), β_(BG) values), shown in FIGS. 13 b and 13 c ,respectively. Using incorrect water type leads to incorrect colors ofthe restored image, as shown both qualitatively and quantitatively bythe zoom-in on the color chart marked by a yellow rectangle.Qualitatively: the rightmost chart shows a pink hue. Quantitatively: Wemeasured the angles in RGB space between the gray level patches and thegray direction [1,1,1] (one angle per patch). The median angle of thepatches is presented bellow the color charts (cos(θ)=1 is a perfectrestoration). The correct values indicate water type 3 (coastal waters),while the incorrect values are for open ocean waters.

Table 6:

Each color chart has six or seven gray level patches. We sample theaverage value of each patch and calculate the cosine of angles betweenthe patch color and the vector [1,1,1]. The median result is presentedin the table. An ideal restoration would yield 1.

Frames (FIG. 14) Pier (FIG. 14) Rocks (FIG. 15) QPcard- QPcard- QPcard-Method Macbeth 202 Macbeth 202 Macbeth 202 Haze- 0.806 0.806 0.920 0.9470.997 0.984 Lines UDCP 0.724 0.788 0.577 0.782 0.946 0.815 WCID 0.7820.805 0.715 0.952 0.930 0.807 The 0.980 0.981 0.923 0.959 0.999 0.997present technique

We present comparative results of our technique against the followingsingle underwater image restoration methods, which are all based on adark channel prior: a naive white-balance and contrast stretching, UDCP(Drews, P., Nascimento, E., Moraes, F., Botelho, S., Campos, M.:Transmission estimation in underwater single images. In: Proc. IEEE ICCVUnderwater Vision Workshop. (2013) 825-830), WCID (Chiang, J. Y., Chen,Y. C.: Underwater image enhancement by wavelength compensation anddehazing. IEEE Trans. Inage Processing 21(4) (2012) 1756-1769) and thepresent restoration technique. In addition, we include the result of thepresent dehazing technique (denoted Haze-Lines) as a baseline.

Each of these two paper suggests a different method for choosing theveiling-light: in WCID it is chosen as the brightest pixel value amongall local minima in a small neighborhood, while in UDCP it is estimatedby finding the brightest pixel in the underwater dark channel I_(dark)^(UDCP)(x)=min_(y∈Ω(x))[min_(c∈{G,B})(I_(c)(y))]. We manually extract Aby averaging a patch in the image, since we find the suggested methodsoften find bright sand pixels as the veiling-light. The top row of FIG.14 shows the pixel chosen as veiling-light by WCID marked by a redcross, the one chosen by UDCP marked by an orange plus, and therectangle chosen manually marked in yellow. Using the same veiling-lightvalue for all methods resulted in artifacts, hence we show the bestresult for each method, with different veiling-light values (the valuesare given in the supplementary material).

FIG. 14 compares prominent single underwater image restoration methods.Applying a nave contrast enhancement is not enough, since the contrastdegradation caused by the medium is spatially non-uniform. This isevident in the left column, where the farther Barracudas are almostindistinguishable from the background, and in the middle column(Frames), where the structure in the back is hardly noticeable.

In FIG. 14 , The top row shows the original input to the restorationmethods, with veiling-light estimation: using UDCP in orange, using WCIDin red and averaging over a patch in yellow for Haze-lines and theproposed method. The rest of the rows, from top to bottom, show theoutput of a global contrast enhancement of each channel (which affectsthe white-balance), Haze-lines, UDCP, WCID, and the present restorationtechnique.

The methods Haze-Lines, UDCP, and WCID do not restore the color of thesand in the foreground of Frames and Pier correctly, as some areas havea blue-green color-cast while others do not. This phenomenon is anindication of an incorrect wavelength-dependent correction, not a globalwhite balance problem. The red color is attenuated much more than theblue and green, and is not amplified enough by these methods. Thepresent restoration technique is able to compensate for thedistance-dependent attenuation. For example, the Barracudas all havesimilar colors in the output image, regardless of their originaldistance. Similarly, the sand in the foreground of Frames has a uniformcolor.

In addition to the qualitative comparison of the images, we used colorcharts as a quantitative measure. The scenes Frames, Pier and Rockscontain two different color charts, at two different distances fromcamera, in order to validate the quality of the restoration. The medianangle between the gray-level patches and the direction [1,1,1] aresummarized in table 6. The present restoration technique out-performsthe other methods.

FIG. 15 shows an image along with the transmission maps estimated in theprocess for three methods: UDCP, WCID, and the present restorationtechnique. Due to the unconstrained nature of the problem, a prior mustbe used to estimate the transmission. Both UDCP and WCID are based onthe dark channel prior, and estimate the transmission according to:

$\begin{matrix}{{{{\overset{\sim}{t}}_{R}(x)} = \left( {1 - {\omega \cdot {\min\limits_{c}\left\lbrack {\min\limits_{y \in {\Omega(x)}}\left( \frac{I_{c}(y)}{A_{c}} \right)} \right\rbrack}}} \right)},} & {{Eq}.(27)}\end{matrix}$where in WCID the outer minimization is carried over c∈{R, G, B}, and inUDCP over c∈{G, B}.

FIGS. 15I.a to 15II.e show the value of the dark channelmin_(c∈{R,G,B})[min_(y∈Ω(x))(I_(c)(y)/A_(c))] per pixel x. In theseparticular cases, the dark channel assumption does not hold. The top ofthe scenes has no objects, and therefore their transmission should tendto zero. However, they are relatively dark and according to the priorcontain no haze. The bright sand in the foreground has a significantvalue in all color channels, and therefore is estimated to containveiling-light by the prior. In contrast, the non-local prior is able todistinguish the foreground sand from the background. The results shownin FIGS. 15I.b, 15I.c, 15II.b, and 15II.c are not restored properly dueto the wrong estimation of the transmission. The prior of the presentrestoration technique produces better results (FIG. 5 d ). For example,FIG. 15I.d shows that the color of the Sea Goldies fish is restored muchbetter than the other methods.

FIG. 15I.a shows the input image. FIGS. 15I.a thru 15I.d show the outputof three restoration methods: UDCP, WCID, and the present restorationtechnique. FIG. 15I.e shows a step in the transmission calculation usingthe dark channel prior:

${\min_{c \in {\{{R,G,B}\}}}\left\lbrack {\min_{y \in {\Omega(x)}}\left( \frac{I_{c}}{A_{c}} \right)} \right\rbrack}.$FIGS. 15I.f thru 15I.h show the transmission maps estimated during therestoration process, corresponding to FIGS. 15I.b thru 15I.d,color-mapped such that warm colors indicate a high transmission, andcold colors indicate a low transmission. For the present restorationtechnique, we show the transmission of the red channel.

FIG. 16 compares our result to methods proposed in Carlevaris-Bianco,N., Mohan, A., Eustice, R. M.: Initial results in underwater singleimage dehazing. In: IEEE OCEANS. (2010) 18; and Peng, Y. T., Zhao, X.,Cosman, P. C.: Single underwater image enhancement using depthestimation based on blurriness. In: Image Processing (ICIP), 2015 IEEEInternational Conference on. (2015) 4952-4956. We did not have the rawfiles from these two papers, so we used the processed image as input toour technique. JPEG compression artifacts are amplified by therestoration method, however the present restoration technique removesthe blue hue completely from the farther corals, unlike previouslyproposed methods. In this figure, 16 a is the input image, 16 b-d showthe output of three restoration methods: Carlevaris-Bianco et al.(2010), Id., Peng et al. (2015), Id., and the present restorationtechnique, respectively.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a hardware processor to carry outaspects of the present invention.

The computer readable storage medium may be a tangible device that mayretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device havinginstructions recorded thereon, and any suitable combination of theforegoing.

A computer readable storage medium, as used herein, is not to beconstrued as being transitory signals per se, such as radio waves orother freely propagating electromagnetic waves, electromagnetic wavespropagating through a waveguide or other transmission media (e.g., lightpulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire. Rather, the computer readable storage mediumis a non-transient (i.e., not-volatile) medium.

Computer readable program instructions described herein may bedownloaded to respective computing/processing devices (which comprisehardware processor) from a computer readable storage medium or to anexternal computer or external storage device via a network, for example,the Internet, a local area network, a wide area network and/or awireless network. The network may comprise copper transmission cables,optical transmission fibers, wireless transmission, routers, firewalls,switches, gateway computers and/or edge servers. A network adapter cardor network interface in each computing/processing device receivescomputer readable program instructions from the network and forwards thecomputer readable program instructions for storage in a computerreadable storage medium within the respective computing/processingdevice.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).

In some embodiments, a hardware processor that is, for example, amicroprocessor, programmable logic circuitry, a field-programmable gatearray (FPGA), or programmable logic arrays (PLA), may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the hardwareprocessor, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It may be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, may be implemented bycomputer readable program instructions.

These specialized computer readable program instructions may be providedto a microprocessor of a general-purpose computer, special purposecomputer, or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via themicroprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks. These computerreadable program instructions may also be stored in a computer readablestorage medium that may direct a computer, a programmable dataprocessing apparatus, and/or other devices to function in a particularmanner, such that the computer readable storage medium havinginstructions stored therein comprises an article of manufactureincluding instructions which implement aspects of the function/actspecified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, may be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

The invention claimed is:
 1. A method for dehazing a digital image,comprising: operating at least one hardware processor to: convert thedigital image to one or more medium-compensated images that are eachbased on attenuation coefficient ratios of a different water type,wherein the digital image is of an underwater scene; cluster hazyRed-Green-Blue (RGB) values of pixels of a digital image into haze-linesin an RGB space, wherein each of the haze-lines is comprised of asub-group of the hazy RGB values that are scattered non-locally over thedigital image; estimate, based on the haze-lines, a transmission map ofthe digital image, wherein the transmission map encodes scene depthinformation for each pixel of the digital image; and calculate a dehazeddigital image based on the transmission map, by recovering true RGBvalues, wherein performing the clustering, the estimating, and thecalculating is for each of the one or more medium-compensated images. 2.A method for dehazing a digital image, comprising: operating at leastone hardware processor to: cluster pixels of a digital image intohaze-lines, wherein each of the haze-lines is comprised of a sub-groupof the pixels that are scattered non-locally over the digital image;estimate, based on the haze-lines, a transmission map of the digitalimage, wherein the transmission map encodes scene depth information foreach pixel of the digital image; and calculate a dehazed digital imagebased on the transmission map, wherein the clustering of pixelscomprises: representing color differences between the pixels of thedigital image and the airlight value on a pre-computed tessellation ofthe unit sphere, where the pre-computed tessellation is uniformlysampled and stored in Cartesian coordinates in a KD-tree; searching fornearest neighbors on the KD-tree using Euclidean distance coordinates;grouping the color samples based on the nearest neighbors, therebyproducing multiple groups each being one of the haze-lines.
 3. Themethod according to claim 2, further comprising, prior to the clusteringof pixels: converting the digital image to one or moremedium-compensated images that are each based on attenuation coefficientratios of a different water type, wherein the digital image is of anunderwater scene; and performing the clustering, the estimating, and thecalculating for each of the one or more medium-compensated images. 4.The method according to claim 2, further comprising computing anestimated global veiling-light value by: generating an edge map of eachof the plurality of medium-compensated images; thresholding the edgemap, to produce multiple pixel blobs; and determining that a color or anaverage color of pixels making up a largest one of the multiple pixelblobs, is the global veiling-light value.
 5. The method according toclaim 2, further comprising: for each of the dehazed digital images:performing global white balancing of a restored digital image, to outputa white-balanced image, calculating a mean color of non-veiling-lightpixels; and outputting the white-balanced image having the closest meancolor to gray.
 6. The method according to claim 2, wherein theestimating of the transmission map comprises: estimating an initialtransmission map as the quotient, for each individual pixel of thepixels of the digital image, of: (a) a distance of the individual pixelfrom an airlight value, and (b) a distance from a pixel which isfarthest away from the airlight value and belongs to the same haze-lineas the individual pixel; and regularizing the initial transmission mapby enforcing a smoothness of the digital image on the initialtransmission.
 7. A system comprising: an image sensor configured toacquire a digital image; a non-transitory computer-readable storagemedium having stored thereon program instructions to: cluster pixels ofthe digital image into haze-lines, wherein each of the haze-lines iscomprised of a sub-group of the pixels that are scattered non-locallyover the digital image, estimate, based on the haze-lines, atransmission map of the digital image, wherein the transmission mapencodes scene depth information for each pixel of the digital image, andcalculate a dehazed digital image based on the transmission map; and atleast one hardware processor configured to execute the programinstructions, wherein the program instructions are further to: prior tothe clustering of pixels: converting the digital image to one or moremedium-compensated images that are each based on attenuation coefficientratios of a different water type, wherein the digital image is of anunderwater scene; and performing the clustering, the estimating, and thecalculating for each of the one or more medium-compensated images. 8.The system according to claim 7, wherein the program instructions arefurther to compute the estimated global veiling-light value by:generating an edge map of each of the plurality of medium-compensatedimages; thresholding the edge map, to produce multiple pixel blobs; anddetermining that a color or an average color of pixels making up alargest one of the multiple pixel blobs, is the global veiling-lightvalue.
 9. The system according to claim 7, wherein the programinstructions are further to: for each of the dehazed digital images:perform global white balancing of the restored digital image, to outputa white-balanced image, calculate a mean color of non-veiling-lightpixels; and output the white-balanced image having the closest meancolor to gray.
 10. The system according to claim 7, wherein theclustering of pixels comprises: representing colors of the pixels of thedigital image in a spherical coordinate system whose origin is anestimated global airlight value; uniformly sampling the unit sphere ofthe representation, to output a plurality of color samples eachassociated with one of the pixels of the digital image; and grouping thecolor samples based on their θ (Theta) and φ (Phi) angles in thespherical coordinate system, according to a mutual closest point on theunit sphere, thereby producing multiple groups each being one of thehaze-lines.
 11. The system according to claim 7, wherein the estimatingof the transmission map comprises: estimating an initial transmissionmap as the quotient, for each individual pixel of the pixels of thedigital image, of: (a) a distance of the individual pixel from anairlight value, and (b) a distance from a pixel which is farthest awayfrom the airlight value and belongs to the same haze-line as theindividual pixel; and regularizing the initial transmission map byenforcing a smoothness of the digital image on the initial transmission.12. The method according to claim 1, further comprising computing anestimated global veiling-light value by: generating an edge map of eachof the one or more medium-compensated images; thresholding the edge map,to produce multiple pixel blobs; and determining that a color or anaverage color of pixels making up a largest one of the multiple pixelblobs, is the global veiling-light value.
 13. The method according toclaim 1, further comprising: for each of the dehazed digital images:performing global white balancing of a restored digital image, to outputa white-balanced image, calculating a mean color of non-veiling-lightpixels; and outputting the white-balanced image having the closest meancolor to gray.
 14. The method according to claim 1, wherein theestimating of the transmission map comprises: estimating an initialtransmission map as the quotient, for each individual pixel of thepixels of the digital image, of: (a) a distance of the individual pixelfrom an airlight value, and (b) a distance from a pixel which isfarthest away from the airlight value and belongs to the same haze-lineas the individual pixel; and regularizing the initial transmission mapby enforcing a smoothness of the digital image on the initialtransmission.